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Overview 


Introduction 


Researchers and policymakers in the early care and education (ECE) field are interested in 
understanding the factors that contribute to successful quality improvement (QI) initiatives in ECE 
settings. They also want to learn about factors leading to improved outcomes for children and families 
through successful QI initiatives. One factor posited to influence the success of such initiatives is 

the readiness of individuals and organizations to adopt new quality improvement practices (Kirk, 
Wanless, & Briggs, 2017; Maxwell, Partika, Wanless, Pacchiano, Halle, Hsueh, & Maher, 2018). In this 
brief, “readiness” is defined as a characteristic of a person, group of individuals, or organization 

that captures at a particular point both the wi/lingness and the capacity to take on a new practice 

or perform an existing practice in a new way. The measurement of readiness within ECE studies is 

still not widespread, partially due to a lack of access to readiness measures tailored for use in ECE 
settings and with early childhood professionals. However, ECE researchers have begun to develop new 
readiness tools and adapt measures of readiness from other fields of study to address this gap. 


Purpose 


The purpose of this brief is to provide a framework for understanding readiness within the ECE field 
and to share examples of how ECE researchers are currently attempting to capture the dimensions 
of readiness—and factors that support readiness—using different data collection methods and 
standardized measurement tools. While not an exhaustive list of measurement options, the summary 
tables provided are meant to share resources that ECE researchers and policymakers may consider 
when preparing to implement new QI initiatives or when developing future studies of QI initiatives. 
We conclude by considering several conceptual and measurement issues that might be addressed by 
further measurement development in the ECE field. 
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Methods 


We identified measures of readiness being used in ECE research and evaluation through a series of 
discussions with ECE researchers and evaluators who are working on various QI projects in states and 
nationally. (See Appendix C for OPRE-funded QI projects that contributed to this activity.) We also 
identified measures of readiness through follow-up online literature searches. In addition, we gathered 
psychometric information about measures through email and phone contact with researchers and 
measures developers between June 2018 and November 2018. 


Key Findings and Highlights 


The review of measures highlights several insights about the current state of readiness measures used 
in ECE studies of quality improvement initiatives. 


e Although the definition of readiness is composed of two components (willingness and capacity), 
we found only one measure that captures both elements. 


e Eight readiness measures have been developed explicitly for use in early care and education 
settings and with the ECE workforce. 


e Several readiness measures have been adapted from other fields. The psychometric properties of 
any new or modified readiness measures should be assessed within the ECE context. 


e There appear to be fuzzy boundaries between direct measures of readiness and factors associated 
with readiness (collectively referred to in this brief as factors affecting readiness). More work is 
needed both conceptually and empirically to distinguish readiness from other related constructs. 
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Introduction 


Researchers and policymakers in the early care and education (ECE) 
field are interested in understanding the factors that contribute to 
successful quality improvement (QI) initiatives in ECE settings. They 
also want to know more about improved outcomes for children 

and families through successful QI initiatives. One factor posited to 
influence the success of such initiatives is the readiness of individuals 
and organizations to take on new QI practices (Kirk, Wanless, & 
Briggs, 2017; Maxwell, Partika, Wanless, Pacchiano, Halle, Hsueh, 

& Maher, 2018; Wanless & Domitrovich, 2015). Understanding and 
measuring readiness to change in ECE settings can help researchers 
and policymakers identify some of the factors that may facilitate Cor 
hinder) the success of an initiative throughout its implementation. 
Program implementers may be able to use information about readiness 
to tailor their support for individuals and teams and make adjustments 
to the implementation of a QI intervention to achieve the desired 
outcomes. 


Despite its importance, few studies currently include readiness as 

a distinct construct within a QI initiative’s theory of change. As a 
result, few researchers intentionally include measures of readiness 
in their studies of program or policy effectiveness. Part of this gap 
is due to a lack of knowledge about and availability of measures of 
readiness deemed appropriate for use in ECE settings. The purpose 
of this brief is to raise awareness of the concept of readiness and to 
share information about possible measurement tools and methods 


Purpose of this brief 


The goal of sharing a 
summary of measurement 
options for assessing 
readiness at the individual 
and organizational levels at 
this juncture is to provide 
a resource that the ECE 
research community can 
build upon in the future. 
However, given the 

limited information on the 
psychometric properties 
of many of these measures 
(some of which have been 
adapted for use in the ECE 
context), researchers and 
policymakers are urged 

to exercise caution in 

their selection and use of 
readiness measures. 


that could be used to capture readiness at multiple levels of the ECE system. We also discuss issues 


relevant for future measurement development. 


This brief provides an overview of definitions of readiness, noting distinctions between individual 
readiness, organizational readiness, and system readiness. We also share examples of how ECE 
researchers are currently attempting to capture the dimensions of readiness, and factors that support 
or hinder readiness, through different data collection methods and standardized measurement tools. 
While not an exhaustive list of measurement options, the summary tables provided share resources 
that ECE researchers and policymakers may want to consider when developing future studies of 
quality improvement initiatives. We conclude this brief by considering several conceptual and 
measurement issues that might be considered when preparing to implement new QI initiatives in the 


ECE field. 


Why Care about Readiness? 


Across multiple disciplines, readiness to change has been theorized as an important component 
contributing to the success of interventions and quality improvement efforts. The study of readiness 
to change has a long history in the fields of business (Cameron & Green, 2009; Rafferty et al., 2013), 


health care (Prochaska, DiClemente, & Norcross, 1992; Weiner, Amick, & Lee, 2008), and occupational 
and organizational psychology (Herscovitch & Meyer, 2002), but has received relatively less attention 
in early childhood (Peterson, 2013). However, with the recent proliferation of QI initiatives in the field 
of early childhood, readiness to change has become more salient as a potential explanatory element 

for understanding the outcomes of QI initiatives. 


Currently, the role of readiness in the effectiveness of QI initiatives in the early childhood field has a 
stronger theoretical basis than an empirical one. Nevertheless, a few recent studies demonstrate the 
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usefulness of considering readiness to change when examining the implementation of QI initiatives. 
For example, Roberts and colleagues reported that readiness to change, along with the related 
factor of psychological anxiety, was associated with multiple indicators of “responsiveness” to an 
online coaching intervention among early childhood educators (Roberts, LoCasale-Crouch, DeCoster, 
Hamre, Downer, Williford, & Pianta, 2015). Furthermore, in this same study, readiness to change was 
found to moderate the association between classroom characteristics and early childhood educators’ 
responsiveness to the coaching intervention (Roberts et al., 2015). 


However, another study by Wanless and colleagues found that a series of measures collectively 
thought to capture “readiness to implement with fidelity” (e.g., observed emotional support, teacher- 
rated use of intervention practices, teacher-rated self-efficacy, teacher-rated collective responsibility, 
etc.) was not directly related to observed measures of fidelity of implementation two years later 
(Wanless, Rimm-Kaufman, Abry, Larsen, & Patton, 2015). Although not in the early childhood field, 

a study by Jones and colleagues tested and found support for the theoretical linkages between 
employees’ perceptions of organizational culture and individual levels of readiness to change, which 
were in turn predictive of “implementation success” of the initiative (Jones, Jimmieson, & Griffiths, 
2005). This same study also reported that pre-implementation levels of readiness for change among 
employees exerted a positive main effect on employees’ satisfaction post-implementation. These 
studies suggest that readiness to change can be relevant to both intervention fidelity and impact, 
either directly or indirectly, although there is some variability in the influence of readiness on these 
outcomes. 


Defining Readiness at Multiple Levels 


In this brief, “readiness” is defined as a characteristic of a person, group of individuals, or organization 
that captures at a particular point both the willingness and the capacity to take on a new practice 

or perform an existing practice in a new way. This definition indicates that readiness involves two 
important subcomponents: a wi/lingness to engage in a new activity, and the capacity to participate 
in a new activity, given existing resources (Peterson, 2013).' Furthermore, willingness and capacity 

to change are, in turn, affected by multiple contextual factors: beliefs and attitudes, social systems 
and relationships, current and persistent stressors, and personal or organizational characteristics (see 
Figure 1). We call these, collectively, factors affecting readiness. 


1’ We acknowledge that other researchers categorize the dual subcomponents of readiness as pertaining to cognitive and affective aspects 
of readiness (Rafferty, Jammieson, & Armenakis, 2013). 
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Figure 1. Conceptual Model for Readiness and Factors Affecting Readiness 
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As Peterson (2013) notes, readiness is also understood to manifest within individuals, organizations, 
and systems. Individual readiness is defined as “the extent to which an individual or individuals are 
cognitively and emotionally inclined to accept, embrace, and adopt a particular plan to purposefully 
alter the status quo” (Holt, Armenakis, Feild, & Harris, 2007, p. 255). Organizational readiness has 

been defined as “organizational members’ change commitment and self efficacy to implement 
organizational change” (Weiner, Amick, & Lee, 2008, p. 68). System readiness is the holistic state of 
preparedness of a system to meet a situation and carry out a planned sequence of actions. Within an 
ECE context, “system” refers to how multiple ECE settings and organizations work together to provide 
services. System readiness requires assessment of the readiness of each component part of the system 
and the level of integration of the component parts within the system (Austin & York, 2015). 


In some QI initiatives, such as those that use a cohort model for professional development or 

engage interdisciplinary collaborative teams in their intervention, it is also important to consider 

team readiness. Team readiness captures the idea that “cognitions and affects of individuals ... 
become shared because of social interaction processes and ...manifest as higher level collective 
phenomena?” (Rafferty, Jimmieson, & Armenakis, 2013, p. 116). It is believed that team readiness can 
feed into organization-wide readiness for change; team and organizational readiness also interact with 
individual readiness (Rafferty et al., 2013). 


Although system readiness and team readiness are important to consider when implementing a 
quality improvement effort, for the remainder of this brief, we focus on individual and organizational 
readiness in particular.? Within an ECE context, individual readiness is assessed among early 
childhood educators, directors, and support staff; in some cases, parents of participating children 
may also be assessed for readiness. Organizational readiness addresses readiness from a program- 
wide perspective, aggregating data from multiple employees’ perceptions of the organization’s 
capacity to support change. An organization’s capacity to support change includes the organization’s 
infrastructure, material resources, and leadership support for the new activity; it also includes 
employees’ perceptions of the overall “climate” of the organization in terms of commitment to quality 
improvement and support for staff to make individual and collective changes to practice. 


2 The study of improvement of ECE systems (e.g., multiple ECE organizations) is not as prevalent as the study of how individuals and 
organizations make improvements in quality or practice; therefore, we limit our review to measures of individual and organizational 
readiness, which are of primary interest to ECE researchers and policymakers at this point. Measures of team readiness would be relevant 
at this time; however, measures of this construct were not located. 
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Measuring Readiness at the Individual and 
Organizational Levels in ECE Settings 


Early childhood researchers are eager to identify valid and reliable measures of individual and 
organizational readiness for use in their studies of QI initiatives. However, there are few tools that 
have been developed specifically for early childhood settings, and even among tools that have been 
developed in other fields, few have evidence of strong psychometric properties (Weiner, Amick & Lee, 
2008). Given the early stages of measuring readiness within ECE initiatives, we offer here a sampling 
of how researchers are currently attempting to measure these constructs within the early care and 
education field. The goal of sharing a summary of measurement options for assessing readiness at 
the individual and organizational levels at this juncture is to provide a resource for the ECE research 
community that can be built upon in the future. However, given the limited information on the 
psychometric properties of many of these measures (some of which have been adapted for use in the 
ECE context), researchers and policymakers are urged to exercise caution in their selection and use 
of measures. By sharing this information, we also hope to spur further discussion about how best to 
include measures of readiness in ECE studies of QI initiatives. 


Methods 


Readiness measures identification and classification 


We identified measures of readiness being used in ECE research and evaluation through a series of 
discussions with ECE researchers and evaluators who are working on various QI projects in states 

and nationally, as well as through follow-up online literature searches. The identified measures were 
then assessed within the framework of readiness presented in this brief organized by willingness and 
capacity and the four categories of factors affecting readiness (see Appendix A), and by individual 
and organizational readiness (see Appendix B). We gathered additional psychometric information 
about measures through email and phone contact with researchers and measures developers between 
June 2018 and November 2018. 


Orientation to the readiness measurement summaries 


We offer two summaries of readiness measurement in ECE in appendices A and B, respectively. 
Appendix A is the Readiness Measures Matrix. The summary tables in Appendix A are organized 
according to the two components of readiness and the four categories of factors affecting readiness 
shown in Figure 1 in this brief. Within each category, specific indicators are identified, and data 
sources currently being used by ECE researchers to capture those indicators are noted. Many of these 
measurement methods are being used, or considered for use, in data collection efforts for recent 
OPRE-funded projects aimed at evaluating QI initiatives. (See Appendix C for further information 
about these OPRE-funded QI projects.) 


While Appendix A is organized by indicator of readiness, Appendix B is organized by measurement 
tool. Appendix B contains an Annotated Bibliography of Standardized Readiness Measures that 
includes all cited measures in Appendix A. The tools are organized alphabetically for both measures 
of individual readiness and organizational readiness. In addition to the full name of the measure and a 
citation, additional information is provided, including the purpose of the measure, constructs covered 
by the measure, and number of items contained in the measure. Where possible, we provide some 
psychometric information about each measure. 


It is important to note that these summaries of measurement options in appendices A and B are not 


3 See Appendix C for OPRE-funded QI projects that contributed to this activity. 
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exhaustive and do not connote endorsements. They were compiled over a series of conversations 
with ECE researchers and evaluators working on various QI projects in states and nationally, and 

the compilations are being offered here to share information with the field as a resource. Individual 
researchers should review all possible measures and determine for themselves which one best meets 
their needs. They should also exercise caution with regard to the limited psychometric information 
available about some of these measurement options.4 


Findings and Discussion 


Initial observations regarding readiness measures used in ECE 


An examination of the information in Appendix A indicates that there are very few readiness measures 
currently being used in ECE research that measure the dual constructs of willingness and capacity. 
The subcomponent of capacity appears to be captured through qualitative interviews with ECE 

staff, review of administrative data, or the development of survey items particularly geared toward 
information about organizational resources and preparedness for a particular initiative. Qualitative 
methods are also often used to capture factors affecting readiness, such as beliefs and attitudes and 
personal and organizational characteristics associated with readiness. There are ample quantitative 
measures that capture the more affective readiness component of willingness, as well as beliefs and 
attitudes, social systems and relationships, and current and persistent stressors that affect readiness. 


Inspection of Appendix A also indicates that some measures are not used exclusively to determine 
readiness directly; they also measure constructs that could be considered factors affecting readiness. 
An example is the Psychological Safety Survey (Edmondson, 1999), which is identified as both a 
measure of openness to change (i.e., an indicator of the component of willingness) and a measure 

of beliefs and attitudes associated with readiness to change. Indeed, psychological safety has been 
claimed as both an e/ement of readiness (Wanless, 2018) and a factor that co-occurs with readiness 
(Wanless, Shafer, & Davis, 2018). The fuzzy boundaries between direct measures of readiness and 
correlates of readiness is a topic we will consider in more detail in the next section of this brief. 


Another observation gleaned from reviews of appendices A and B is that, although several measures 
currently being used by ECE researchers have been adapted from related fields, such as education and 
mental health (e.g., Bay Area School Reform Collaborative Teacher Survey Scale, Stanford University, 
2002; Teacher’s Efficacy Scale, Hoy & Woolfolk, 1993; University of Rhode Island Change Assessment 
Scale, McConnaughy, Prochaska, & Velicer, 1983), at least eight measures have been developed 
explicitly for use in early learning settings and with the ECE workforce (Bloom, 2010; Bloom, 2015; 
Bloom, 2017; Ehlrich et al., 2018; Horsley & Fong, 2017; Peterson, Baker & Weber, 2010; VandeWiele, 
2001; Wanless, 2014). For example, an ECE setting’s readiness for change can be assessed using the 
Early Childhood Work Environment Survey (Bloom, 2010) or the Assessment of a School’s Readiness 
to Change (Wanless, 2014). Similarly, leadership in ECE settings (considered a factor affecting 
readiness) can be measured by both the Director’s Role Perception Survey (Bloom, 2017) and the 
Preschool Instructional Leadership Scale (Horsley & Fong, 2017). 


Further considerations for conceptualizing and measuring readiness 
in ECE 


Recent interest in the role of individual and organizational readiness in the success of early childhood 
quality improvement initiatives has spurred the development and use of readiness measures in studies 


4 As noted above, measures developers were contacted between June 2018 and November 2018 to obtain psychometric information on 
measures for which this information was not otherwise readily accessible. Researchers were contacted about psychometric information on 
adapted measures during this same time period. The notation in Appendix B indicating psychometric information is “not available” does 
not necessarily mean that this information does not exist; it means that the authors of this brief were not able to obtain the information to 
include in the brief by November 2018. This designation is distinct from the notation of “none,” which indicates that the authors definitively 
know that no psychometric information is available on a particular measure or adapation of a measure. 


iM CekeK1Ul a lave Mnxs¥-Lellal=s-tin xe) am @¥at-lale (=m lal =t-1d N/a Ot-lc-¥r-lale ml =fe[Ulor-la loyal) 


of ECE quality improvement initiatives, as indicated in the appendices of this brief. Yet, measurement 
of readiness in ECE settings is still new, and a consideration of conceptual and methodological issues 
with the current state of readiness measurement is warranted. 


Conceptual issues 


As noted above, there seems to be some ambiguity regarding whether a particular measurement 
instrument is a direct measure of readiness or is measuring distinct factors that are associated 

with readiness, such as psychological safety, leadership, or organizational climate. Where does 

the construct of readiness end and these other related constructs begin? Weiner and colleagues 
(2008) propose that readiness to change should be specific to a particular change referent (for 
example, a particular quality improvement effort), and that a general capacity to change is better 
conceptualized as organizational culture or climate. Indeed, some ECE studies currently use measures 
of organizational climate, for example, as a proxy for organizational readiness for change (see 
Appendix C). Other researchers, however, claim that it is possible to measure an organization’s general 
capacity to change separate from a specific capacity to change a particular practice or process 

(Holt, Armenakis, Harris, & Feild, 2007; Meyers, Durlak & Wandersman, 2012; Scaccia, Cook, Lamont, 
Wandersman, Castellow, Katz, & Beidas, 2015). It is possible that quality improvement efforts in ECE 
will require both a general measure of readiness as well as a measure specific to the change referent. 
For example, if the quality improvement effort is focused on improving math instruction in early 
childhood settings, it may be important to assess individual and organizational readiness for taking on 
the specific math activities, in addition to assessing the organization’s general capacity to take on any 
new instructional activities. 


Current efforts in developing measures of readiness in ECE settings have focused on readiness to 
support social and emotional learning (Maxwell, Partika, Wanless, Pacchiano, Halle, Hsueh, & Maher, 
2018). Future work should clarify whether there are meaningful distinctions between measures of 
readiness to change and measures of characteristics closely associated with readiness to change. 
Furthermore, future research should explore the level of specificity needed to understand readiness. 
Such research should address the extent to which readiness is dependent on what programs are trying 
to get ready for (Wanless & Domitrovich, 2015). 


Another conceptual issue that deserves further consideration is the association between the different 
levels of readiness. For example, how does organizational readiness support change for an individual, 
and how does individual or team readiness support an organization’s overall capacity for change? 
Some theoretical work describes interactions between individual and organizational readiness as 
well as the relationships among antecedents and outcomes of readiness (Rafferty et al., 2013), but 
we do not yet have conceptual models that situate these relationships within the broader context 

of implementation of a quality improvement initiative. It is possible that readiness plays either a 
mediating or moderating role in achieving quality improvement outcomes, similar to what has been 
found for the related construct of psychological safety (Edmondson & Lei, 2014). Future conceptual 
work is needed to better articulate the role of both individual and organizational readiness within a 
theory of change for quality improvement efforts in ECE settings. 


Measurement issues 


Although the definition of readiness is composed of two components (willingness and capacity), there 
is no one quantitative measure that captures both elements. Rather, most measures capture either 

the psychological component of wi/lingness to change or the behavioral component operationalized 
by an individual’s or an organization’s perceived capacity to implement change (Weiner et al., 2008). 
For the sake of parsimony, it would be helpful to have a single measure that captures both elements 
of readiness. However, given that it is likely that separate measures will be needed to capture both 
individual and organizational readiness, researchers will likely need to consider using multiple 
readiness measures in their research regardless. 


Another consideration is the timing of data collection on readiness. It is important to note that 
although readiness is often considered during the planning or early implementation stages of an 
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initiative, it should be assessed throughout the entire life course of an initiative and not just at major 
milestone decision points (Austin & York, 2015). However, not all measures are designed to be used 
at all stages of the change process. Researchers should consider whether a quantitative readiness 
measure they choose is sensitive to change over time and whether their research design permits 
assessment of readiness at multiple stages of implementation. Researchers should also consider the 
use of qualitative methods, such as focus groups, key informant interviews, and document review for 
ongoing monitoring of readiness, and factors associated with readiness throughout the life of a Ql 
initiative. 


In addition, several of the measures noted in this brief have been adapted from other fields. As such, 
the psychometric properties of any new or modified readiness measures should be assessed within the 
ECE context. 


Finally, researchers should also consider whether and how to operationalize system readiness in 
early care and education. As several significant quality improvement initiatives are targeted at the 
systems level (e.g., state early childhood professional development systems, state quality rating and 
improvement systems), it seems appropriate to consider capturing system readiness for QI initiatives. 
Further conceptual and measurement development is needed in this area. 


Further considerations for ECE practice 


Beyond measuring readiness for research purposes, these measures are important for guiding and 
customizing the implementation of a QI effort (Weiner et al., 2008). How individuals or programs 
perform on a readiness measure at a given point provides important information to technical 
assistance providers who can use this information to tailor their support for effective implementation 
of a new practice. Measures of readiness should not be used to determine whether a program or 
individual is “ready” for a new practice or way of conducting an existing practice, but rather as a tool 
that can determine goodness of fit of a particular QI intervention for an organization, or the level and 
nature of tailored support an individual or organization needs to engage successfully in a particular QI 
initiative. In sum, readiness information can be used to indicate that an individual, group of individuals, 
or the entire organization needs a higher level of support for the current intervention, or a different 
type of intervention altogether. 


Readiness measures can also provide contextual information for understanding fidelity to a new 
practice at the individual, team, and/or organizational levels, and can serve as a barometer for the 
“implementation environment” conducive to the take-up and spread of evidence-based interventions 
(Bumbarger, 2015). When readiness measures are incorporated into implementation and impact 
evaluations, they can help researchers, program developers, and policymakers understand findings 
regarding the quality of program implementation as well as program effectiveness and impact. 
Future implementers and other stakeholders should aim to use readiness information to support ECE 
programs and a diverse ECE workforce in their efforts to improve quality practices and better child 
outcomes (Peterson, 2013). 


Conclusion 


The concept of readiness to change has received little attention within the field of early care and 
education until recently (cf. Kirk, Wanless, & Briggs, 2017; Maxwell, Partika, Wanless, Pacchiano, Halle, 
Hsueh, & Maher, 2018; Wanless, Groark, & Hatfield, 2015). The measurement of readiness within ECE 
studies is still not widespread, partially due to lack of access to readiness measures tailored for use 
in ECE settings and with early childhood professionals. However, ECE researchers have begun to 
develop new readiness tools and adapt measures of readiness from other fields of study to address 
this gap. This brief summarizes measurement tools and strategies that ECE researchers are currently 
using to capture both individual and organizational readiness for quality improvement initiatives in 
early childhood settings. We note that at least eight standardized measures have been developed for 
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use in ECE settings. While not an exhaustive list, we share this information as a resource to the field 
and in the hopes of encouraging further development, validation, and use of readiness measures in 
ECE research and program evaluation, and in the implementation of QI initiatives. This information 
may also encourage further conceptual work to incorporate readiness into our theories of change for 
quality improvement in early childhood programs, policies, and systems. 
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Appendix A. Readiness Measures Matrix 


Readiness consists of two dimensions—willingness and capacity (Peterson, 2013). An individual’s or 
organization’s willingness and capacity to change is affected by multiple contextual factors: beliefs 
and attitudes, social systems and relationships, current and persistent stressors, and personal or 
organizational characteristics. We call these factors, collectively, factors affecting readiness. These 
may, in turn, be measured through various indicators using quantitative or qualitative methods. The 
summary tables in this appendix are organized according to the two categories of readiness and the 
four categories of factors affecting readiness shown in Figure 1 in this brief. Within each category, 
specific indicators are identified, and data sources currently being used or considered by ECE 
researchers to capture those indicators are noted. Many of these measurement methods are being 
used, or considered for use, in data collection efforts for recent OPRE-funded projects aimed at 
evaluating QI initiatives. See Appendix C for further information about these projects. 


It is important to note that the summaries of measurement options in this appendix are not 

exhaustive and do not connote endorsements. They were compiled over a series of conversations 

with ECE researchers and evaluators working on various QI projects in states and nationally, and 

the compilations are being offered here to share information with the field as a resource. Individual 
researchers should review all possible measures and determine for themselves which one best meets 
their needs. They should also exercise caution with regard to the limited psychometric information 
available about some of these measurement options (see Appendix B for psychometric information on 
measures). 


Table Al. Readiness Components: Willingness & Capacity 


Aaerpae Readiness 7 ' Data Sources 
late | Ket- 1 xe) g eve 
a Qualitative Quantitative 
Psychological Safety Survey 
(Edmondson, 1999) 
Interview 
with program Assessment of a School’s 
CIS INEES Se Willingness Organizational | directors Readiness for Change (Wanless, 
change 2014) 
Readiness for Change Scale (Holt, 
Armenakis, Feild & Harris, 2007) 
Stage of Change Scale for Early 
Education and Care 2.0 (Peterson, 
Baker, & Weber, 2010) 
ed a Willingness Individual 
g Adapted University of Rhode 
Island Change Assessment 
CURICA, McConnaughy, Prochaska, 
& Velicer, 1983) 
Interview 
with program Survey items 
directors 
Cones Capacity | Organizational | Review of 
capacity application to 
participate in Ql 
Survey items 
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Table A2. Beliefs & Attitudes 


to professional 
development 


Organizational 


Beliefs about Interview Adaptation of the Teachers’ Attitudes about 
professional Individual with teachers, Professional Development (TAP; Torff, Sessions, 
development directors & Byrnes, 2005) 
Beliefs around Social and Emotional Learning 
s atvielual survey (adapted from Brackett et al., 2012) 
: ndividua 
Beliefs about Adapted Teacher Beliefs Scale for Coaches 
target of the Ql (Burts et al., 2000; Charlesworth et al., 1993) 
initiative 
Srasnisstionel Assessment of a School’s Readiness for Change 
g (Wanless, 2014) 
eee Early Childhood Work Environment Survey 
with program 
; (Bloom, 2015) 
directors 
Early Learning Mentor Coach (ELMC) Survey 
. (Howard et al., 2011) 
Commitment Review of 


application to 
participate in 
Ql 


Assessment of a School’s Readiness for Change 
(Wanless, 2014) 


Head Start CARES End-of-Year Reflections 


Goals for 
participation in 
the QI initiative 


Individual 


QI participant 
focus group 


Survey items 


Adaptation of LA Advance Questionnaire 


Organizational 


Review of 
application to 
participate in 
Ql 


Interview 
with program 
directors 


Survey items 


Ideas to address 


Review of 
application to 


anticipated Organizational participate in 
challenges Ql 
Survey items 
Inquiry Organizational Psychological Safety Survey (Edmondson, 1999) 
practices : . : : 
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Table A2, cont. Beliefs & Attitudes 


Motivation for 
participation in 
the QI initiative 


Individual 


Interview 
with program 
directors 


Survey items 
Interview 


with program 
directors 


Survey items 


Adaptation of LA Advance Questionnaire 


Review of 
application to 
participate in 


alignment of QI 
with personal 
goals 


Individual 


Perceived Ql . ; 
. Readiness for Change Scale (Holt, Armenakis, 
alignment ‘ : 
of initiative Organizational | Interview Feild & Harris, 2007) 
with other QI with program 
priorities directors 
QI participant 
focus group 
Perceived 


Survey items 


Psychological 
safety 


Organizational 


Psychological Safety Survey (Edmondson, 1999) 


Q-CCIIT Caregiver Self-Administered 
Questionnaire 


Early Childhood Work Environment Survey 
(Bloom, 2015) 
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Table A3. Social Systems & Relationships 


Collaboration 


Organizational 


Adapted Psychological Safety Survey 
(Edmondson, 1999) 


School Conditions—Teacher Learning 
Community Subscale, Bay Area School 
Reform Collaborative Teacher Survey 
Scale (BASRC; Stanford University, 
2002) 


Early Ed Essentials (Ehlrich et al., 2018) 


Survey of Organizational Functioning 
Cohesion Subscale (Texas Christian 
University Institute of Behavioral 
Research, 2008) 


Commitment 
to goals/shared 
vision 


Organizational 


Early Childhood Work Environment 
Survey (Bloom, 2015) 


Communication 


Organizational 


Survey of Organizational Functioning 
(Texas Christian University Institute of 
Behavioral Research, 2008) 


Organizational 
culture 


Organizational 


Early Childhood Work Environment 
Survey (Bloom, 2015) 


Items from NCES SASS 2011-2012 
Teacher Survey 


LA Advance Administrator Survey 
Early Ed Essentials (Ehlrich et al., 2018) 


Q-CCIIT Caregiver Self-Administered 
Questionnaire 


Leadership 


Organizational 


Review of 
application to 
participate in QI 


Individual 


Chicago Consortium for School 
Research Teacher Survey 


Preschool Instructional Leadership 
Scale (PILS; Horsley & Fong, 2017) 


Early Ed Essentials (Ehlrich et al., 2018) 


Local/state 
policies to 
support 
intervention topic 


Organizational 


Assessment of a School’s Readiness for 
Change (Wanless, 2014) 


Positive 
community 
relationships 


Organizational 


Assessment of a School’s Readiness for 
Change (Wanless, 2014) 
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Bay Area School Early Childhood Work Environment 


Staff Reform Collaborative | Survey (Bloom, 2015) 
relationships/ Organizational | Teacher Survey Scale 
support (Stanford University, | Assessment of a School’s Readiness for 
2002) Change (Wanless, 2014) 
Early Childhood Job Satisfaction Survey 
Trusting (Bloom, 2010) 
relationship : : 
with director, Individual Adapted LA Advance Questionnaire 


supervisor, other Early Ed Essentials (Ehlrich et al., 2018) 
staff, and coach 


UPCOS-5 Teacher Interview 


Table A4. Current & Persistent Stressors 


DE) t= Lol Ul cox=1-) 


Take ket=h ke) a3 
Qualitative Quantitative 
Individual Maslach Burnout Inventory—Educator’s 
Survey (Maslach, 1997; Maslach, Jackson, 
& Leiter, 1996) 
Burnout/stress 
Kessler Distress Scale (Kessler & 
ks Mroczek, 1992; Kessler et al., 2002) 
Organizational 
Assessment of a School’s Readiness for 
Change (Wanless, 2014) 
Administrative data 
Job benefits Organizational Early Childhood Work Environment 
Survey (Bloom, 2015) 
Staff salaries Organizational Survey items 
Administrative data 
Staffing/ O oii 
turnover Panizatlona Items from ECE-ICHQ Center Director 
Questionnaire 
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Table A5. Personal & Organizational Characteristics 


Take lCet=h Ke) a3 


Allocation of time 
for PD 


Level 


Organizational 


Data Sources 


Qualitative 


Interview with 
program directors 


Review of application 
to participate in Ql 


Survey items 


Quantitative 


Early Childhood Work 
Environment Survey 
(Bloom, 2015) 


Survey items 


Content knowledge 
related to the QI 
initiative 


Individual 


Adapted Beliefs about 
Infant-Toddler Education 
and Care (BAITEC; 
adapted version by 
Valloton, based on 
Anderson, 2015) 


Knowledge of Infant 
Development Inventory 
(KIDI, MacPhee, 1981) 


Organizational 


Review of application 
to participate in Ql 


Interview with 
program directors 


Assessment of a School’s 
Readiness for Change 
(Wanless, 2014) 


Data systems & use 
of data 


Organizational 


Review of application 
to participate in Ql 


Interview with 
program directors 


Survey items 


Head Start CARES 
End-of-Year Reflections 


Education and 
training qualifications 


Individual 


Q-CCIIT Caregiver 
Self-Administered 
Questionnaire 


Surveys of program staff 


Early Learning Mentor 
Coach (ELMC) Survey 
(Howard et al., 2011) 
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Table A5, cont. Personal & Organizational Characteristics 


Take lets ke) 3 


Resources available 
for planning 


Level 


Organizational 


Data Sources 


Qualitative 


Interview with 
program directors 


Discussions with Ql 
facilitators 


Quantitative 


Early Learning Mentor 
Coach (ELMC) Survey 
(Howard et al., 2011) 


Head Start CARES Coach 
Trainer Log 


Adapted LA Advance 
Questions 


Q-CCIIT Caregiver 
Self-Administered 
Questionnaire 


Professional educator 
identity 


Organizational 


Assessment of a School’s 
Readiness for Change 
(Wanless, 2014) 


Individual 


Interviews 


Self-efficacy 


Individual 


Segments of Early 
Childhood Teaching 
Inventory CECTI, 
VandeWiele, 2001) 


Segments of Directors’ 
Role Perception Survey 
(Bloom, 2017) 


Stage of Change Scale for 
Early Education and Care 
2.0 (Peterson, Baker, & 
Weber, 2010) 


Teachers’ Sense of 
Efficacy Scale or 
adaptations (Hoy & 
Woolfolk, 1993) 


Organizational 


Assessment of a School’s 
Readiness for Change 
(Wanless, 2014) 


Structural center 
characteristics (é.g., 
venue type, child age 
range served, hours 
of operation, and 
others) 


Organizational 


Review of application 
to participate in QI 


Interview with 
program directors 


Successful 
implementation 
history 


Organizational 


Assessment of a School’s 
Readiness for Change 
(Wanless, 2014) 


Technology literacy 


Individual 


Adaptation of 
Administrator Technology 
Survey, 2014 
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Appendix B. Annotated Bibliography of 
Readiness Measures 


This appendix provides basic information about readiness measures at the individual and 
organizational levels, corresponding to measures noted in Appendix A of this brief. It is important 

to note that the summaries of measurement options in this appendix are not exhaustive and do not 
connote endorsements. They were compiled over a series of conversations with ECE researchers and 
evaluators working on various QI projects in states and nationally, and the compilations are being 
offered here to share information with the field as a resource. Individual researchers should review all 
possible measures and determine for themselves which one best meets their needs. They should also 
exercise caution with regard to the limited psychometric information available about some of these 
measurement options.§ 


Individual Level Measures 


Adaptation of the Administrator Technology Survey (Burris, 2014) 


e Purpose: To assess caregiver’s comfort with using technology and online platforms 
e Constructs: comfort using various technologies 

e Length: 14 items 

°¢ Reliability: none 

°¢  NValidity: none 


Available at: http://udspace.udel.edu/bitstream/handle/19716/16711/2014 Burris Jade DEd. 
pdf?sequence=1&isAllowed=y 


Adapted Beliefs about Infant-Toddler Education and Care (BAITEC; adapted version by Valloton, 
based on Anderson, 2015) 

e Purpose: To examine infant-toddler teacher belief as an indicator of child care quality 

e Constructs: teachers’ beliefs about developmentally appropriate education and care 

e Length: 38 items 

¢ Reliability: The author reported high reliability of the measure (Anderson, 2015). 

¢ MNalidity: The author reported good content, convergent, and construct validity (Anderson, 2015). 


Original measure available at: https://docs.lib.ourdue.edu/cgi/viewcontent. 
cgi?article=1233&context=open_access dissertations 


5 As noted in the Methods section of this brief, measures developers were contacted between June 2018 and November 2018 to obtain 
psychometric information on measures for which this information was not otherwise readily accessible. Researchers were contacted about 
psychometric information on adapted measures during this same time period. The notation in this annotated bibliography indicating that 
psychometric information is “not available” does not necessarily mean that this information does not exist; it means that the authors of 
this brief were not able to obtain the information to include in the brief by November 2018. This designation is distinct from the notation 
of “none,” which indicates that the authors definitively know that no psychometric information is available on a particular measure or 
adaptation of a measure. 
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Beliefs around Social and Emotional Learning survey (adapted from Brackett et al., 2012) 


e Purpose: To assess teachers attitudes and support for SEL 

e Constructs: comfort, commitment, culture 

e Length: 10 items 

e Reliability: Reliability of all variables exceed .7 (Collie, Shapka, Perry, & Martin, 2015). 
e  Malidity: none 


Original measure available at: http://ei.yale.edu/wp-content/uploads/2016/05/Brackett-et-al.-Teacher- 
SEL-Beliefs-Scalel.pdf 


Chicago Consortium for School Research 5Essentials Teacher Survey (University of Chicago, 2018) 
e Purpose: to identify strengths and weaknesses at the district and school level and better target 
resources and interventions 


e Constructs: effective leaders, collaborative teachers, ambitious instruction, supportive 
environment, involved families 


e Length: 45 minutes 

e Reliability: Individual and school level reliability for most measures exceeds .7 

e WValidity: The authors reported evidence for the validity of this measure (Klugman, Gordon, Sebring, 
& Sporte, 2015) 


Available at: httos:/www.uchicagoimpact.org/sites/default/files/2018%20CPS%205Essentials%20 
Teacher%20Codebook.pdf 


Director’s Role Perception Survey, items related to self-efficacy (Bloom, 2017) 


e Purpose: To assess ECE directors’ self-efficacy 

e Constructs: Director’s self-efficacy in leadership, management, and supporting children’s learning 
e Length: 35 items 

¢ Reliability: none 

°«  Nalidity: none 


The tool is in the pilot phase with the McCormick Center for Early Childhood Leadership at National 
Louis University. 


Early Childhood Job Satisfaction Survey (Bloom, 2010)* 

e Purpose: To assess child care workers’ satisfaction with their jobs 

e Constructs: Trusting relationship with director, supervisor, other staff, and coach 
e Length: Approximately 15 minutes 


e Reliability: The author conducted reliability tests in three separate samples and found the overall 
internal consistency of the measure ranged from .81 to .9 between the samples. The authors also 
found high test-retest reliability for each subscale (Bloom, 2016). 


¢  MNalidity: The author reported high face validity, convergent validity, and discriminant validity 
(Bloom, 2016). 


Available at: http://newhorizonsbooks.net/assessment-tools-2/early-childhood-job-satisfaction-survey/ 


iM Ce¥oK1Ul a lave Mnx-¥-Lellal=s-t-¥n xe) am @¥at-lale (=m lal =t-1d hymn Ot-lc-¥r-lale mm =fe[Ulor-la loyal) 


Early Childhood Teaching Inventory (VandeWiele, 2001) 


e Purpose: To assess teachers’ self-perceptions of their abilities in the early childhood environment 
e Constructs: responsivity, focus, professional knowledge, crisis management 

e Length: 38 items 

¢ Reliability: none 

°¢ Validity: none 


Measure is not readily available but was used in a past OPRE-funded study (Quality Intervention for 
Early Care and Education; QUINCE). 


Kessler Distress Scale (Kessler & Mroczek, 1992; Kessler et al., 2002) 


e Purpose: To measure stress 
e Constructs: Burnout, stress 


e Length: 6 or 10 items—6 items were included in the National Survey of Early Care and Education 
(https:/www.acf.hhs.gov/opre/resource/workforce-classroom-staff-questionnaire-revised) 


e Reliability: Internal consistency reliability of the original KIO and K6 was .96 and .89, respectively, 
in a telephone pilot sample (Kessler et al., 2002). 


e Validity: The K10 was included in the 1997 NSMHWB and the K6 was included in the 1997 NHIS. 
These data were analyzed against the pilot IRT results to cross-validate the KIO and K6 measures 
(Kessler et al., 2002). 


Available at: https://www.tac.vic.gov.au/files-to-move/media/upload/k10_english.pdf 


Knowledge of Infant Development Inventory (KIDI, MacPhee, 1981) 


e Purpose: To assess the accuracy of parents’ and child care workers’ knowledge of infant and child 
development 


e Constructs: Accurate knowledge of infant and child develooment 
e Length: Approximately 15 minutes 


e Reliability: Internal reliability of the scale for a sample of parents (a description of the parents was 
not provided by the authors) was .82, and test-retest reliability over a 2-week period for the same 
sample of parents was .91 for the number of items accurate (MacPhee, 1981). 


« Validity: MacPhee (1981) reported good content validity of the scale, and evidence of construct 
validity. 


This tool is available by contacting the author: David.Macphee@ColoState.EDU 


Maslach Burnout Inventory - Educator’s Survey (Maslach, 1997; Maslach, Jackson, & Leiter, 1996) 


e Purpose: To measure burnout among those working in educational settings 
e Constructs: Burnout, stress - (emotional exhaustion, depersonalization, personal accomplishment) 
e Length: 22 items 


e Reliability: All subscales of the MBI, except for the Depersonalization (Factor IV) subscale, were 
determined to be reliable for use with teachers (lwanicki & Schwab, 1981). 


e  MNalidity: In their analysis of the construct validity of the educator survey, Ilwanicki and Schwab 
(1981) found the MBI measured the same constructs as the original MBI. 


Available at: httos:/www.mindgarden.com/117-maslach-burnout-inventory 
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Preschool Instructional Leadership Survey (Horsley & Fong, 2017) 

e Purpose: To measure the frequency of instructional leadership behavior 

e Constructs: effective leadership, professional capacity, instructional guidance 
e Length: 17 items 


¢ Reliability: The author reported item reliability greater than .9, supporting high reliability of this 
measure (H. Horsley, personal communication, November 29, 2018) 


e Validity: The author reported that Rasch analysis supported the validity of this measure (H. Horsley, 
personal communication, November 29, 2018) 


This tool is available by contacting the author: hhorsley@mail.fresnostate.edu 


Stage of Change Scale for Early Education and Care 2.0 (Peterson, Baker, & Weber, 2010) * 


e Purpose: To assess teachers’ readiness to change 


e Constructs: Stages of change (pre-contemplation, contemplation, preparation, action, 
maintenance); attitudes/beliefs about quality improvement/change; teacher self-efficacy 


e Length: 7-item teacher survey, 7-item coach survey 


° Reliability: Test reliability of the scale was .93 for a sample of home visitors/mentors from the 
Partners in Family Child Care (PFCC) project (Montes, Peterson, & Reynolds, 2011) 


* Validity: The authors reported concurrent and predictive validity with the Child/Home 
Environmental Language and Literacy Observation (Montes et al., 2011). 


Available at: https://www.childrensinstitute.net/ 


Teachers’ Attitudes about Professional Development (Torff, Sessions, & Byrnes, 2005) 

e Purpose: To assess the extent of teachers’ support for professional development initiatives 
e Constructs: Beliefs about professional development 

e Length: 5 items 


e Reliability: The authors reported high internal consistency reliabilities of the measure (Torff, 
Sessions, & Byrnes, 2005). 


¢  MValidity: The authors reported satisfactory construct and discriminant validity of the measure 
compared to other related measures (Torff et al., 2005). 


Available at: http://journals.sagepub.com/doi/abs/10.1177/0013164405275664 


Adapted Teacher Beliefs Scale for Coaches (Burts et al., 2000) 


e Purpose: To measure teacher beliefs and attitudes toward accepted practices in preschool 
settings 


e Constructs: Developmentally appropriate practice, child-initiated practice, didactic 
e Length: 15 items 

¢ Reliability: not available 

¢  Nalidity: not available 


Visit the Family and Child Experiences Survey (FACES) website: https:/www.acf.hhs.gov/opre/ 
research/project/head-start-family-and-child-experiences-survey-faces for more information on this 
measure. 
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Teachers’ Sense of Efficacy Scale (Hoy & Woolfolk, 1993) 


e Purpose: To assess teachers’ self-efficacy 


e Constructs: Teacher self-efficacy in Student Engagement, Instructional Practices, and Classroom 
Management 


e Length: 24 items 


e Reliability: For a sample of elementary school teachers, reliability of personal teaching efficacy 
and general teaching efficacy was .77 and .72, respectively (Hoy & Woolfolk, 1993). 


e MNalidity: The authors reported good convergent and discriminant validity of the scale, based on 
previous analyses (Hoy & Woolfolk, 1993). 


Available at: https://www.jstor.org/stable/1002017?seq=l#page scan_tab contents 


University of Rhode Island Change Assessment Scale (URICA; McConnaughy, Prochaska, & Velicer, 


1983) 
e Purpose: To assess readiness to change, originally in the context of substance abuse and mental 
health 


e Constructs: Stages of change (pre-contemplation, contemplation, preparation, action, 
maintenance) 


e Length: 31 items 
¢ Reliability: not available 
¢ Validity: not available 


Available at: httos://www.ncbi.nim.nih.gov/books/NBK64976/table/A62309/ and at https://web.uri. 
edu/cprc/psychotherapy-urica/ 


UPCOS-5 Teacher Interview (Moiduddin, Aikens, Sprachman, Atkins-Burnett, Winston, & Smith, 2011) 


e Purpose: gather information on key elements of the coaching process 


e Constructs: relationship with coach; goal-setting; coaching strategies and activities; content of 
coach interactions 


e Length: 47 items 
° Reliability: none 
°« Validity: none 


Available at: http://www.first5la.org/files/O7110_401.1Design&AnalysisPlan_Final_10062011.pdf 


*Can be aggregated and used as a measure of organizational readiness 
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Organizational Level Measures 
Adaptation of LA Advance Questionnaire and Administrator Survey 


e Purpose: to assess the coach-teacher relationship 


e Constructs: curricula used, type of professional development (PD) received, PD topics, hours per 
month of PD 


e Length: 4 items 
¢ Reliability: none 
°«  Validity: none 


Further information available by contacting Emily Moiduddin at Mathematica: emoiduddin@ 
mathematica-mpr.com 


Assessment of a School’s Readiness for Change - Teacher Version (Wanless, 2014) 


e Purpose: To assess a school’s readiness for change 

° Constructs: Readiness at the community, school, principal/director, and teacher level 

e Length: 24 items 

e Reliability: Inter-item reliability of the assessment was .899 (Kirk, Wanless, & Briggs, 2017) 
¢ Validity: not available 


This tool is available by contacting the author: swanless@pitt.edu 


Early Childhood Work Environment Survey (Bloom, 2015) 


e Purpose: To assess the organizational climate of an early childhood program 


e Constructs: Psychological safety; organizational culture; commitment to professional development; 
allocation of time for professional development; commitment to goals/shared vision; openness to 
change 


e Length: Approximately 15 minutes 


¢ Reliability: The author conducted reliability tests in two samples and found the overall internal 
consistency of the measure was .93 and .95, respectfully. The authors also found evidence for test- 
retest reliability for the measure (Bloom, 2016). 


¢  MNalidity: The author reported high concurrent validity and discriminant validity of the measure 
(Bloom, 2016). 
Available at http://newhorizonsbooks.net/assessment-tools-2/early-childhood-work-environment- 


survey/ 


Early Ed Essentials (Ehrlich, Pacchiano, Stein, & Wagner, 2018) 
e Purpose: To measure organizational supports essential for change within school- and community- 
based ECE settings 


e Constructs: Effective instructional leaders, ambitious instruction, collaborative teachers, involved 
families, parent voice, and supportive environment 


e Length: 100 items for staff survey, 54 items for parent survey 


¢ Reliability: The authors reported that most measures on the staff and parent surveys scored above 
.8, suggesting high reliability of both surveys (Ehrlich et al., 2018b). 


e  Malidity: The authors reported acceptable internal validity on all measures (Ehrlich et al., 2018b). 


Visit the project website (https://www.theounce.org/what-we-do/research/programs/five-essentials- 
early-ed-surveys/) for more information about accessing the tool. 
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Early Learning Mentor Coach (ELMC) Survey (Howard et al., 2011) 


e Purpose: To assess coaches’ commitment to professional development 


e Constructs: Background and experience of coaches; preparation for the ELMC initiative; the 
approach to coaching; the goals and the content for coaching; any perceptions about the 
effectiveness of coaching; challenges and facilitating factors about coaching 


e Length: Approximately 30 minutes 
e Reliability: none 
°  Malidity: none 


Available at https://www.acf.hhs.gov/sites/default/files/opre/early learning mentor_coach_elmc 
coach survey 508.pdf 


Head Start CARES Coach Trainer Log 

e Purpose: To assess the teacher’s view of the relationship with the coach 
e Constructs: teachers’ experience with coaching 

e Length: 3 items 

° Reliability: none 

¢ Validity: none 


Further information available by contacting Emily Moiduddin at Mathematica: emoiduddin@ 
mathematica-mpr.com 


Items adapted from the ECE-ICHQ Center Director Questionnaire 

e Purpose: To obtain information from ECE directors about their organization 

e Constructs: staffing structure, minimum education requirements for staff 

e Length: 7 items about staffing; 5 items about minimum education of different staff 

¢ Reliability: none 

°¢  WNalidity: none 

Available at https://www.reginfo.gov/public/do/PRAView!C?ref_nbr=201501-0970-002&icID=218410 


NCES SASS 2011-2012 Teacher Survey, items related to organizational culture (Cox, Parmer, Strizek, 
& Thomas, 2017) 


e Purpose: To obtain information from teachers about organizational culture 

e Constructs: class organization 

e Length: 12 items 

e Reliability: The authors reported good internal consistency reliability (Wolfe, Ray, & Harris, 2004). 
e Validity: The authors reported evidence of structural validity of the measure (Wolfe et al., 2004). 


Available at: https://nces.ed.gov/surveys/sass/pdf/1112/SASS4A. pdf 
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Psychological Safety Survey (Edmondson, 1999) 
e Purpose: To assess the organizational climate of an early childhood program 


e Constructs: Psychological safety; openness to change; knowledge seeking; collaboration; problem- 
solving 


e Length: 7 items 


¢ Reliability: The author reported adequate internal consistency reliability on most measures. 
Reliability was low for context support, .65, and team efficacy, .63 (Edmonson, 1999) 


e Validity: The author did not establish discriminant validity but did establish construct and face 
validity of some measures (Edmondson, 1999). 


Available at http://journals.sagepub.com/doi/abs/10.2307/2666999 


Readiness for Change Scale (Holt, Armenakis, Feild & Harris, 2007) 

e Purpose: To assess readiness for change 

e Constructs: Alignment of initiatives with other QI priorities 

e Length: 59 items 

e Reliability: Reliability exceeded .7 on all scales except for Personal Valence (.66 and .65; Holt, 
Armenakis, Feild, & Harris, 2007) 

e MNalidity: The authors reported convergent validity for all scales, some evidence of predictive 
validity, and incremental validity for the readiness factors (Holt et al., 2007). 

Available at http://journals.sagepub.com/doi/abs/10.1177/0021886306295295 


School Conditions—Teacher Learning Community Subscale, Bay Area School Reform Collaborative 
Teacher Survey Scale (Stanford University, 2002) 


e Purpose: To understand the teacher learning community 
* Constructs: Collaboration; staff relationships/support 
e Length: 5 items 


e Reliability: Reliability on the Teacher Learning Community Subscale exceeded .8 across four years 
of reporting (2002, 2003, 2004, 2005). 


¢ Validity: not available 
Available at https://crceducation.stanford.edu/system/files/basrc-teacher-survey-scale2005 O.pdf 


Survey of Organizational Functioning (Texas Christian University Institute of Behavioral Research, 
2008) 


e Purpose: To assess job attitudes, workplace practices, and organizational readiness for change 
e Constructs: Collaboration; communication 

e Length: 129 items, but can be broken down into multiple domains 

e Reliability: not available 

¢ Validity: not available 

Available at: http://ibr.tcu.edu/wp-content/uploads/2013/06/SOF-sg.pdf 
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Q-CCIIT Caregiver Self-Administered Caregiver Questionnaire (Atkins-Burnett, Monahan, Tarullo, 
Xue, Cavadel, Malone, Akers, 2015) 


Purpose: to gather information about caregivers’ experience and the structural features of 
classrooms and family child care settings 


Constructs: classroom characteristics, caregiver characteristics 
Length: 31 items 

Reliability: none 

Validity: none 


Available at: https:/www.acf.hhs.gov/sites/default/files/opre/measuring the quality of caregiver 
child interactions for infants and O.pdf 
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Appendix C. OPRE Project Descriptions 


This appendix includes summaries of current projects funded by the Office of Planning, Research, 
and Evaluation that are considering the construct of readiness in their studies of QI initiatives. The 
following are highlights of overall projects, as well as brief summaries of measures beings used to 
capture readiness at the individual and/or organizational levels. 


Culture of Continuous Learning (CCL) Project: A Breakthrough Series Collaborative for 
Improving Child Care and Head Start Quality, 2016-2019 


https:/www.acf.hhs.gov/opre/research/project/creating-a-culture-of-continuous-quality- 
improvement-in-child-care-and-head-start-settings 


This project explores how child care and Head Start programs can improve the quality 
of services young children receive, while institutionalizing continuous quality improvement 
activities. The project assesses the feasibility of implementing a continuous quality 
improvement (CQI) methodology, the Breakthrough Series Collaborative (BSC), to promote 
the uptake and sustainability of evidence-based practices for social and emotional learning 
(SEL) in both child care and Head Start settings. 


The BSC is a unique method designed to improve the uptake, sustainability, and spread of 
evidence-based practices. A BSC includes five key elements: (1) the Change Framework; 

(2) Multi-Level Inclusive Teams; (3) Expert Faculty; (4) a Shared Learning Environment; and 
(5) the Model for Improvement. BSC teams meet regularly and include program directors, 
teachers, and parents. The BSC groups are designed to create a shared learning environment 
in which CQI strategies are used to test research-based practices and make adjustments 
based on short-term, informal data collection. The goal is to influence changes in the culture, 
climate, structures, and leadership within ECE settings as well as the knowledge, skills, 
beliefs, and attitudes of practitioners participating in the BSC. 


In 2018, a BSC focused on supporting children’s SEL within eight ECE settings (4 Head 
Start, 4 child care) in an East Coast, urban area began. A study to assess the feasibility of 
implementing a BSC is being conducted alongside implementation to better understand 
whether a BSC can successfully improve SEL practices in ECE programs. The project uses 

an embedded case study design and data from multiple sources at multiple timepoints, 
across all phases of implementation of the BSC. The feasibility study aims to understand 
organizational and individual characteristics that relate to feasibility, and the supports within 
the BSC that are associated with progress towards improvement. 


The project’s individual level readiness measures include the Early Job Satisfaction 
Survey, Teachers’ Sense of Efficacy Scale, Director’s Role Perception Survey, Maslach 
Burnout Inventory, and their Team Selection Questionnaire. The project’s organizational 
level readiness measures include the Psychological Safety Survey, Early Childhood Work 
Environment Survey, and their Team Selection Survey. 


The project is being conducted through a contract with Child Trends and its subcontractors 
the University of Massachusetts-Boston, JRA Consulting, Ltd., and the Center for the Study 
of Social Policy. 
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Development of a Measure of the Quality of Caregiver-Child Interactions for Infants and 
Toddlers (Q-CCIIT), 2010-2014 


https:/www.acf.hhs.gov/opre/research/project/develooment-of-a-measure-of-the-quality-of- 
caregiver 

The goal of this project was to develop a new measure to assess the quality of child care 
settings, specifically the quality of caregiver-child interactions for infants and toddlers in 
non-parental care. The new measure is sensitive to the developmental levels of children ages 
birth to 3 years as well as to adult behaviors that are responsive and stimulating for this 

age group. In addition, it is appropriate for use across different types of child care settings 
(i.e., center-based and family child care homes) as well as for both single- and mixed-age 
classrooms. Key tasks included (1) assessing the state of the measurement field related to 
child-adult interactions and quality of care settings for infants and toddlers; (2) developing 
a measure to assess the quality of child-caregiver interaction in infant-toddler care settings; 
and (3) conducting psychometric testing to ensure the soundness of the measure for diverse 
populations and settings, including settings serving low-income families, settings having 
ethnic/racial diversity in caregivers and children, settings with children from non-English 
speaking households, and settings with children with disabilities. 


The project’s individual level readiness measures include the Q-CCIIT Caregiver Self- 
Administered Questionnaire, Stages of Change Self-Report, adapted LA Advance 
Questionnaire, UPCOS-5 Teacher Interview, KIDI, CES-D-10 Short Form Scale, Kessler 6 Self 
Report Measure, selected items from BAITEC, new items from the Q-CCIIT Program Director 
Questionnaire, and adaptations of the Administrator Technology Survey and the Teacher 
Opinion Survey. The project’s organizational level readiness measures include the Q-CCITT 
Caregiver Self-Administered Questionnaire, new items from the Q-CCIIT Program Director 
Questionnaire, and an adaptation of LA Advance Administrator Survey. 


Study of Coaching Practices in Early Care and Education Settings (SCOPE), 2016-2019 


https:/www.acf.hhs.gov/opre/research/project/study-of-coaching-practices-in-early-care-and- 
education-settings 


This contract was awarded to Mathematica Policy Research in partnership with Child 
Trends and the Children’s Learning Institute at the University of Texas Health Science 
Center at Houston. The study is exploring how coaching practices are implemented and 
vary in ECE classrooms serving children supported by Child Care and Development Fund 
(CCDF) subsidies or Head Start grants. This project aims to advance understanding of 
how core features of coaching are implemented in ECE classrooms, how they may vary 
by key contextual factors, and which are ripe for more rigorous evaluation. Tasks include: 
(1) establishing an empirically supported conceptual model for the project, (2) designing 
and conducting a descriptive study to examine the occurrence and variability of coaching 
features in ECE classrooms, and (3) conducting case studies to examine program- or 
systems-level drivers of coaching and the features being implemented. 


The project’s individual level readiness measures includes adaptations of the Early Learning 
Mentor Coach (ELMC) Survey, the Family and Child Experiences Survey (FACES), LA 
Advance Questionnaire, Head Start CARES Coach Trainer Log, and the Early Childhood 
Teacher Survey. The project’s organizational level readiness measures include the ECWES 
Short Form Teacher Survey and adaptations of items from ELMC, the National Survey of 
Early Care and Education (NSECE), LA Advance, and FACES. 
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Variations in Implementation of Quality Interventions: Examining the Quality-Child 
Outcomes Relationship in Child Care and Early Education (VIQI), 2016-2021 


https:/www.acf.hhs.gov/opre/research/project/variations-in-implementation-of-quality- 
interventions-examining-the-quality-child-outcomes-relationship 


The VIQI study is testing how different levels and features of classroom quality relate to 
children’s developmental outcomes. The study examines the relationship between initial 
child care and early education (CCEE) classroom quality and changes in observed quality 
and children’s outcomes through a rigorous experimental design. 


Questions about the quality-child outcomes relationship will be addressed in the context of 
an in-depth implementation study to determine the conditions necessary to plan, install, and 
implement an evidence-based intervention that will produce changes in process, domain- 
specific quality and child outcomes. CCEE classrooms will include those in Head Start, child 
care and public-K programs serving children ages 2-4, not yet in kindergarten. The study is 
being conducted through a contract with MDRC and its subcontractors Abt Associates/Abt 
SRBI, Frank Porter Graham Child Development Institute, and MEF Associates. 


The project’s individual level readiness measures include the Stages of Change Scale 2.0, 
Teachers’ Sense of Efficacy Scale, Maslach Burnout Inventory, and an adaptation of the 
Teacher Beliefs Scale. Its organizational level readiness measures include the NCES SASS 
2011-2012 Teacher Survey, Chicago Consortium for School Research Teacher Survey, ECE 
ICHQ Center Director Questionnaire, Assessment of a Preschool’s Readiness for Change, and 
Readiness for Change Scale. 
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